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ABSTRACT 


Approximately 42 percent of the Harcum Junior College 



student body participated in a study of faculty effectiveness. 
Results of the Faculty Evaluation Check List, which consisted of 10 
Likert-type items and two open-ended questions, indicate that 
students have high regard for their professors. A. copy of the 
instrument is included. (AG) 
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1. "A fundamental assumption relating to evaluation is that such evaluation of faculty 
instruction through some form of rating, or measurement device, is of value. "(Blai, 
p. 153) The purpose of this particular Institutional Research Report (IRR 71-37) is to 
provide the faculty and administration with individually unidentifiable student opinions 
on various aspects of faculty teaching performance, and to present a collective 
'picture* of faculty effectiveness as viewed by a very substantial -sized sample 

of die Harcum student bo(fy. "The items ... in the Faculty Evaluation Check-list 
are offered solely because they appear, in my judgment, to have "face validity" as 
items reflecting teacher behavior which can contribute to die learning of students. 

They are not offered as items selected through "item analysis"; "factor analysis, " 
or other sophisticated statistical techniques. They simply seem to 'make sense.'" 
(Blai, p. 155) 

2. "The growing body of research literature on the use of student ratings of teachers 
indicates that they can and do make reasonable, accurate ratings (Simpson, 1966). 

As Howsan (1960) reports: their ratings tend to agree with each other, and the teachers 
who are rated best by the pupils tend to obtain the highest pupil gains . Pupil ratings 
often do not agree with die ratings by principals, supervisors or teachers . (This 
has not been considered as an indication of weakness in student ratings, however, 
since ratings by supervisors and peers have not been shown to agree with pupil- 
gained measures or to be satisfactory^ in other ways.) Teachers have indicated their 
belief that pupil ratings, as obtained in research studies, are bodi fair and acciucate." 
(Blai, p. 154) 
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3. The very sizeable student-population samples, selected completely at random, vdio 
participated in this pilot evaluation during academic year 1970-71, is rejtlected in the 
following tabulation: 

Division Average size of class samples 



Business 


sa 


43% 


Fine Arts 


ts 


42% 


Education 


= 


40% 


English & Language 


a 


40% 


Science & Math 


a 


43% 


Social Sciences 


s 


47% 



Clearly tihen, the comments wdiich follow reflect the concensus of views of a substantial 
number (approximately 270) of the Harcum student population of 630 . 
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4. A useful method of comparing experimentally obtained results with those to be 
expected dieoretically on the hypothesis of equal probability (the so-called "null 
hypodiesis) is the Chi-square (X^) test. Application o^ diis formula permits one 
to determine if statistic^y significant differences exist between 'theoretical* and 
'obtained* results. The more closely the observed results approximate the expected, 
the smaller the chi-square, and the closer die agreement between observed data and 
the hypothesis being tested. Contrariwise, the larger the chi-square, the greater 
the probability of a real divergence of experimentally observed from expected results. 
The equation for chi-square is:- 

" (fo-fe)^ 



X2 



fe 



in which:- fo = frequency of occurrence of 'observed* facts 

fe = "expected" theoretical frequency of occurrence 
5.'»=sumof 

5. Theoretically, an equal number of faculty (20%), through chance variation alone 
(equal probability) would be 'rated* in each of the five adjective categories - Superior, 
Above Average, Average, Below Average , Poor. This assumption would be applicable 
for each of the 10 items included in the Faculty Evaluation Check List utilized . The 
"observed" ratings, in contrast to die theoretical "expected" ratings, are summarized 
in the following tabulation. All percentages reported are rounded off to the nearest 
vhole number . 



Faculty Evaluation Chedc List Ratings 



Item 



Superior 



Above 

Aver. 



Aver. 



Below 

Aver. 



Poor 



1. Instructor's preparation? 


48% 


29% 


18% 


3% 


2% 


2. Fairness? 


38% 


32% 


21% 


7% 


2% 


3 . Knowledge of subject? 

4 . Lecture s or discussions follow 


64% 


24% 


8% 


3% 


1% 


organized pattern? 

5 . Clarity of assignments as to what is 


40% 


29% 


21% 


5% 


5% 


expected of students? 

6 . A patterned timetable worked out to fit 


37% 


27% 


23% 


5% 


8% 


tLs course to number of class meetings? 
7 . Skill in communication clarity of 


42% 


27% 


21% 


7% 


3% 


expression and explanation? 


43% 


28% 


19% 


6% 


4% 


8. Helpfulness to individual students? 

9. Emotional stability (even temperament. 


43% 


28% 


18% 


7% 


4% 


patience, and sense of humor)? 

10. Enthusiasm, (ability to "sell" students 
on desirability of their working to 


54% 


25% 


13% 


4% 


4% 


capacity? 


49% 


25% 


18% 


5% 


3% 
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6. Were any of these differences between 'theoretical' and 'observed' frequencies 
statistically significant? Application of the Chi-square formula to the data yielded 
the following results: 

Item #1 - Instructor's preparation:- = 63 .2 with a probability or "P" = less 
than.OOOl. In general, one may safely discard a null hypothesis of "equal probability" 
whenever "P" is .05 or less. Clearly thf n, this distribution of ratings may be marked 
"Significant" at less than the .0001 level (i.e . less than 1 chance in 10,000 of this 
distribution occuring through sampling fluctuations or 'chance' alone). Therefore, 
the null hypothesis is rejected and it is concluded that this substantial sample of 
Harcum students strongly favor the teaching effectiveness of the Harcum faculty as 
reflected in the item - "histructor's preparation for each class (lab session." 

7 . The Chi-square values for each of the other 9 items in the Faculty Evaluation 
Check-List were also found to be "Significant" at less than the .0001 level. Therefore, 
as might be anticipated, the distribution of the total 2187 student evaluations among the 
5-item adjective &cale of Superior (46%), Average (27%), Average (18%), Below 
Average (5%), and Poor (4%) was also found to be statistically significant; to a very 
high level of confidence. It is therefore concluded, with reason, that the Harcum 
faculty, as a group, are by all odds far and above average as teachers , 

8. This conclusion is furth.er supported by the write-in comments of "teacher strong 
points" which were offered by 60% of the 270 student-respondents. Without itemizing 
each of the 162 statements the theme clearly apparent throughout was the fact that 
Harcum faculty are student-oriented; helpM and patient ^vith Individuals and 
enthusiastic! 

9. Equally gratifying is the very short-list of "shortcomings" - the write-in comments 
of "teacher weak points" vdiich consisted of only 55 items (24%) . In die main, these 
spoke of limited or non-existent opportunities for class discussion; lecturing too 
rapidly; not being sufficiently explicit; and expecting too much from students. 

10 . And finally, it was particularly noteworthy that in the space provided on the 
Check-List for writing in "Teacher's weak points", some 41 students wrote - "none"! ' 

11. This was only a limited-scope study, the project's objective being to explore 
insufficiencies and problems. Hopefully diis has been achieved, and it therefore 
augers well for the possible future full administration of some type of "composite" 

(Blai, 1971) faculty evaluation plan, 

/" i 

Boris Blai, Jt. Ed.D. 

Director of Research 

December 1971 Note;- Special thanks are due Ms Margaret 

Flagg, Class of 1972, who processed and 
summarized the student evaluations - the 

basic data in this pilot study . 
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Faculty Evaluation Check List 



Above Below 

Items Superior Average Average Average Poor 

J i 5 - ^ 

1 . Instructor's preparation for each 

class (lab.) session? 

2, Fairness? — 

3« Knowledge of subject? 

4 . Lectures or discussion follow 

organized pattern? 

5 . Clarity of assignments as to ^’d^a^: 

is expected of students? — 

6 . A patterned timetable worked out to fit 

the course t) number of class meetings? 

7 . Skill in communication clarity of 

expression and explanation? — 

8. Helpfulness to individual 

students? . 

9. Emotional stability, (even temperament, 

patience, and sense of humor)? 

10. Enthusiasm, (ability to "sell" students 
on desirability of their working to 

capacity? — 

Teacher's strong points 



Teacher's week points 



Superior 
Above Average 
Average 
Below Average 
Poor 



= Exceptional performance 
s: Greater tiian satisfactory 
g Satisfactory 

= Minimal acceptable performance 
" Non^acceptable performance 



Appendix A 
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